Robust Rankings for College Football

We investigate the sensitivity of the Colley Matrix (CM) rankings—one of six computer rankings used by the Bowl Championship Series—to (hypothetical) changes in the outcomes of (actual) games. Specifically, we measure the shift in the rankings of the top 25 teams when the win-loss outcome of, say, a single game between two teams, each with winning percentages below 30%, is hypothetically switched. Using data from 2006–2011, we discover that the CM rankings are quite sensitive to such changes. To alleviate this sensitivity, we propose a robust variant of the rankings based on solving a mixed-integer nonlinear program, which requires about a minute of computation time. We then confirm empirically that our rankings are considerably more robust than the basic CM rankings. As far as we are aware, our study is the first explicit attempt to make football rankings robust. Furthermore, our methodology can be applied in other sports settings and can accommodate different concepts of robustness besides the specific one introduced here.